Method of generating a digital video image using a wide-angle field of view lens

ABSTRACT

A method of generating a digital video image uses a wide-angle field of view (WFOV) lens positioned closely in front of an image sensor array so that the image field of the lens is so curved at the sensor array that different regions of the image field are substantially in focus on the sensor array for different positions of the lens. The method comprises selecting a desired region of interest in the image field of the lens, and adjusting the lens/array distance to bring the region of interest into focus on the sensor array. The in-focus region of interest is stored and at least partially corrected for field-of-view distortion due to the WFOV lens. The corrected image is displayed, locally and/or remotely. These steps are cyclically repeated to provide the video image.

This invention relates to a method of generating a digital video imageusing a wide-angle field of view (WFOV) lens. The invention isespecially applicable to, but not limited to, the use of a mobile phone(cell phone) or other embedded device or digital image acquisitiondevice, e.g., for video conferencing.

A typical mobile phone has a built-in camera with front and/or rearfacing lenses. The front lens is used when the phone is used as atraditional camera, while the rear facing lens, which is on the sameside of the camera as the viewing screen, is used when the user wishesto make a video call to another user. The typical field of view (FOV) ofthe rear facing lens is a 55 to 60 degree cone centred symmetricallyabout optical axis of the lens. When used for video calling, asillustrated in the example of FIG. 1, the camera 10 is held usually atarm's length in front of the user 12 with its rear lens facing the user(the lenses are not shown since they are usually recessed below thesurface of the phone). As stated, the rear lens has a 55-60 degree FOV14 centred on its optical axis 16.

Such a phone could be conventionally used as a video conferencing deviceby passing the phone from one person to another as the need arises. Thisis unsatisfactory and does not promote a natural interactive conference.

It would be desirable to adapt a mobile phone or indeed any digitalimage acquisition device for video conferencing so that the phone can belaid flat on a table with its rear lens facing upwards. The desired lenswould also have a sufficiently wide-angle field of view (WFOV) that allparticipants sitting round the table are included in the single field ofview, as illustrated in the example of FIG. 2. The distance of the user12 from the phone will depend on the angle of the FOV 14, but typicallythe sitting distance will be about 40 cm for a FOV of 140 degrees, thecut-off point being about 15 cm below the shoulder. However, the use ofa WFOV lens in such an application conventionally would give rise toproblems of focus and distortion in the image captured by the phone.

As to distortion, WFOV lenses typically produce an image which isheavily distorted, especially at its edges where the conferenceparticipants would normally be seated. It would be desirable to have asystem wherein this distortion is at least partially corrected.

The problem of focus is illustrated in FIGS. 3A-3B. FIG. 3Aschematically shows an example of a conventional optical system, where acomposite lens 20 has a normal field of view 14 (typically, e.g., 55degrees) and is relatively distant from the image sensor 22. Althoughthe image field 24 (known as the Petzval surface) is typically curved atthe image sensor 22, the curve is very shallow compared to the plane ofthe sensor, so that the portion of the image field 24 falling on thesensor 22 can be brought substantially into focus over the entire areaof the sensor.

However, in the case of a WFOV lens 20′ positioned close to the sensor22, as in the example illustration of FIG. 3B, as would be the case in amobile phone or similar compact format devices, the image field 24 issignificantly curved relative to the plane of the sensor 22 so that forany given distance between the lens 20 and the sensor 22, a part of theimage field, less than the entire image field, is substantially in focuson the sensor. By moving the lens 20′ towards or away from the imagesensor 22, as indicated by the double-headed arrow in FIG. 3B, differentregions of the image field can be brought substantially into focus onthe sensor. This is shown schematically in the example of FIGS. 4A-4F.

In FIGS. 4A-4F the lens 20′ (not shown) has moved progressively nearerto the image sensor 22. The parallel lines 26 which intersect the imagefield 24 indicate which region of the image field is in focus for thatsetting on the lens. The in-focus region 28 is shown hatched in FIGS.4A-4C. FIGS. 4D-4F illustrate the corresponding positions of the imagefields from FIGS. 4A-4C, respectively, relative to the image sensor ineach case. Thus, in FIG. 4A just the central region of the image fieldhaving radius r is shown to be in focus. In FIG. 4B an annular regionbetween r₀ and r₁ surrounding the central region is in focus. In FIG. 4Can outer annular region is in focus. The lens 20′ is not confined justto move stepwise between the three positions shown. The lens may beconfigured to move progressively and smoothly towards or away from theimage sensor 22. The in-focus region will generally expand or contractmore or less equally smoothly.

It is desired to mitigate these problems both in mobile phones used invideo conferencing and also in other devices where a WFOV lens is placedclosely in front of an image sensor.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates the conventional use of a mobile phone for videocalling.

FIG. 2 illustrates the desired use of a mobile phone for videoconferencing.

FIGS. 3A-3B and 4A-4F illustrate certain problems arising in theimplementation of a mobile phone for video conferencing.

FIG. 5 is a schematic diagram of a video conferencing system inaccordance with certain embodiments.

FIG. 6 shows schematically an exemplary digital image acquisition devicefor acquiring and processing a succession of images according to certainembodiments.

DETAILED DESCRIPTION OF THE EMBODIMENTS

A method is provided for generating a digital video image using a lenspositioned in front of an image sensor array. The lens has asufficiently wide field of view (WFOV), and is positioned sufficientlynear to the sensor array, that the image field of the lens is so curvedat the sensor array that different regions of the image field aresubstantially in focus on the sensor array for different positions ofthe lens relative to the sensor array.

The method in accordance with certain embodiments includes:

-   -   (a) selecting a desired region of interest in the image field of        the lens,    -   (b) adjusting the position of the lens relative to the sensor        array so that the selected region of interest is brought        substantially into focus on the sensor array,    -   (c) capturing and storing the image on the sensor array of the        substantially in-focus selected region of interest,    -   (d) at least partially correcting the stored substantially        in-focus image for field-of-view distortion due to said WFOV        lens,    -   (e) displaying the corrected image, and    -   (f) cyclically repeating steps (a) to (e).

Further embodiments will next be described, by way of example, withreference to the accompanying drawings FIGS. 5-6.

In FIG. 5, a digital image acquisition device, for example, a mobilephone 10 is laid rear-side up flat on a table such as the tableillustrated in side view in FIG. 2. The mobile phone 10 includes adigital image acquisition component including a WFOV lens 20′. The lens20′ may be configured as shown in FIGS. 4A-4F. The lens 20′ has asufficiently wide field of view (FOV), and is positioned sufficientlynear to the sensor array 22, that the image field of the lens is socurved at the sensor array that different regions of the field of viewof the lens image field are substantially in focus on the sensor arrayfor different positions of the lens relative to the sensor array. Thelens 20′ may be an autofocus MEMS lens. These provide for very rapidlens movement for focussing. Examples of such lenses and lens modulesare disclosed in U.S. Pat. No. 8,358,925, US 2012/0119425, and US2012/0119612, which are incorporated by reference. Examples of suchlenses are commercially available from DigitalOptics Corporation under,for example, the model nos. DOC2054AFS, DOC2054AF, DOC3075AF. Theselenses and lens modules incorporating these lenses can in turn beincorporated into image acquisition devices. Examples of devices whichcurrently incorporate sensor arrays of size in accordance with certainembodiments and which could be adapted to include WFOV lenses of theabove type include the Nokia 808 PureView which has a 41 Mpx 1/1.2″sensor.

The video conferencing system further includes an input device 30 suchas a joystick or mouse, and a display monitor 32, both in communicationwith the phone 10, for example by Wi-Fi, Infra-Red, Bluetooth or anyother suitable wired or wireless link. The rectangle 34 within theoutline of the phone 10 represents, not to scale, the image field of thelens 20′ falling on the sensor 22 positioned within the phone bodyimmediately behind the lens. Actually the entire image field of the lensextends beyond the edges of the sensor, but in this discussion we willfocus on the part falling on the sensor. This image field 34 will not bevisible to the participants of the video-conference even if, as might bethe case, it is displayed on the phone's display screen, because thiswill typically be facing down.

When video conferencing mode is selected, the camera software defines adefault region of interest (ROI) 36. In certain embodiments, the defaultROI is positioned centrally in the image field 34, although severalalternative schemes are employed in further embodiments including:picking a ROI surrounding a largest face detected within the imaged FOV;or possibly directing the ROI towards a face which is determined to bespeaking based on either video or audio analysis of the imaged FOV. If acentral ROI is chosen, this may be pointing up at the ceiling in certainembodiments. Using the input device 30, the ROI 36 can be panned acrossthe image field 34, for example to the position 36′.

The input device 30 local to one end of the conference could be used tointeract with the display provided from a remote location to allow usersat one end to manually select a ROI at the remote end. As such, in thiscase, movement of the local remote device 30 is in certain embodimentstherefore relayed to the remote location and/or vice versa. If howevercontrol of the ROI to be displayed is local as in other embodiments, thecommands may be received from a local input device 30.

As the ROI 36 moves across the image field the phone softwarecontinually adjusts the distance of the lens 20′ from the sensor 22 tomaintain the current ROI in focus. Due to the speed with which theposition of the MEMS lens can be adjusted towards and away from thesensor, this is advantageously done substantially in real time.Alternatively, movement of the ROI to a particular location on the imagefield could be governed by: gesture recognition; for example, byrecognising that a person within the field of view has raised his arm;or by face detection, wherein one or more persons within the field ofview is/are detected as candidate regions of interest. In any case, theROI 36 would move to cover such a detected person's head and shoulders.

The part of the image field on the sensor 22 corresponding to theinstantaneous in-focus ROI 36 is successively captured at the videoframe rate and stored for further processing. Such processing includescorrecting each stored in-focus ROI 36 for field-of-view distortionintroduced by the WFOV lens 20′. While such correction may not beperfect, especially as it has to be done once per video frame, itnevertheless tends to render the image advantageously much more true tolife than the heavily distorted image produced by the lens 20′.

Referring to FIG. 6, patent applications such as PCT/EP2011/052970, U.S.application Ser. Nos. 13/077,891 and 13/541,650, which are incorporatedby reference, disclose digital image acquisition devices including WFOVlens systems. Here, distorted WFOV images are read from a sensor via animaging pipeline which can carry out simple pre-processing of an image,before being read across a system bus into a frame buffer in systemmemory for further processing.

Such systems can employ hardware modules or sub-modules also connecteddirectly or indirectly to the system bus for reading successive imagesstored in system memory from the bus and for processing the image beforeeither returning the processed image to system memory or forwarding theprocessed image for further processing by other modules.

In FIG. 6, for example, a WFOV correction module successively readsimage portion(s) bounding areas of the frame buffer corresponding toselected ROIs within the image field and can provide corrected imageportions to one or both of a mixer module and a face detection (FD) andtracking module. In certain embodiments, more than one region ofinterest may be tracked, corrected and/or displayed at any given time.

As explained below, the face detection and tracking module mightalternatively obtain images directly from system memory independently ofthe correction module (as indicated by the dashed line).

A system controller controls the various hardware modules. The systemcontroller is in certain embodiments responsive to, for example,commands received through a control interface from, for example,software applications running on the device with which a user interacts.In this case, the input device 30 (local or remote) may be used.

In FIG. 6, a zoom and pan module is connected to the controller and thisin turn communicates with the WFOV correction module to determine whichpart of an acquired image is to be read from system memory forcorrection. Thus, the zoom and pan module is responsive in certainembodiments, or is configurable to be responsive, either to automaticdetection of candidate region(s) of interest and/or manual selection bya user of a region of interest.

The mixer module can, for example, superimpose user interface icons inthe images of the stream to be displayed on display 32 and so assistwith user interaction.

In any case, the in-focus and distortion-corrected ROIs are transmittedin certain embodiments as successive frames or frame portions of a videosignal to the remote location (i.e., the other end of the videoconferencing connection), as well as to the local monitor 32 for displayas a video image in a reserved part 40 in the upper right of the monitorscreen 42. This is to allow the local participants to check what isbeing sent to the remote location. As mentioned, the rest of the screen42 is typically occupied by the video image from the remote location.

In certain embodiments, it is possible to both pan and zoom the ROI 36,so that an enlarged ROI cannot be brought substantially into focus as awhole on the sensor array 22 for a single lens/array distance. In theseembodiments, the camera software may be configured to sub-divide anenlarged ROI into two or more sub-regions and adjust the lens/arraydistance in respect of each sub-region to bring that sub-regionsubstantially into focus on the sensor array in a number of separateimages. The multiple sub-regions could then be extracted from multipleinput images captured in quick succession with different ROIs in focusand stored, and then combined to form a substantially in-focus image ofthe enlarged region of interest. This synthesised ROI is then in certainembodiments distortion corrected to provide one enlarged ROI for onetime frame of the video signal as previously described. Alternatively,distortion correction may be performed on the individual ROIs extractedfrom respective input images before they are combined to provide theenlarged ROI.

In some implementations, rather than employing a more conventional RGBor RGBW sensor array, an RGBIR (RGB infrared) sensor array can beemployed. This allows separate processing to be performed on separateplanes of the images being captured by the sensor array. So, forexample, the face detection (or gesture detection) module of FIG. 6could operate on a non-distortion corrected, sub-sampled version of thecomplete field of view to identify one or more ROI's corresponding tosubjects who have been detected within the field of view; or possiblysubjects moving within the field of view. As IR (infrared) images tendnot to be completely sharp in any case, it can be sufficient fordetection and tracking purposes to operate on the distorted version ofthe scene in attempting to detect and track potential candidateregion(s) of interest.

The correction module then in turn processes only the RGB planes of theROI to provide the corrected image for the region of interest forsubsequent display.

The invention is not limited to the embodiments described herein whichmay be modified or varied without departing from the scope of theinvention.

US published patent application US 2011-0216156 and U.S. applicationSer. Nos. 13/077,891, 13/078,970 and 13/084,340, which are assigned tothe same assignee and hereby incorporated by reference, disclose digitalimage acquisition devices including WFOV lens systems. In certainembodiments, distorted WFOV images are read from a sensor via an imagingpipeline which is configured to carry out pre-processing of an imagebefore being read across a system bus into system memory.

Such systems can employ hardware modules or sub-modules also connecteddirectly or indirectly to the system bus for reading successive imagesstored in system memory from the bus and for processing the image beforeeither returning the processed image to system memory or forwarding theprocessed image for further processing. The WFOV correction moduleillustrated by example in FIGS. 3A-3B is configured to successively readdistorted images or image portions and provide corrected images or imageportions to a face detection (FD) and/or tracking module.

An efficient mechanism is provided in certain embodiments for performingcomplex distortion compensation on an input image in a processor andmemory in an efficient manner with relatively low or even minimaldemands on the system bus.

Advantageous correction modules are provided herein to addressdistortion problems in images captured by various types of digital imageacquisition devices including digital still cameras, video cameras,camera-phones, camera modules, web cameras, and other camera-enableddevices. All references cited herein are incorporated by reference,including the following as describing camera devices and features inaccordance with alternative embodiments:

U.S. Pat. Nos. 7,224,056, 7,683,468, 7,936,062, 7,935,568, 7,927,070,7,858,445, 7,807,508, 7,569,424, 7,449,779, 7,443,597, 7,768,574,7,593,636, 7,566,853, 8,005,268, 8,014,662, 8,090,252, 8,004,780,8,119,516, 7,920,163, 7,747,155, 7,368,695, 7,095,054, 6,888,168,6,583,444, and 5,882,221, and US published patent applications nos.2012/0063761, 2011/0317013, 2011/0255182, 2011/0274423, 2010/0053407,2009/0212381, 2009/0023249, 2008/0296717, 2008/0099907, 2008/0099900,2008/0029879, 2007/0190747, 2007/0190691, 2007/0145564, 2007/0138644,2007/0096312, 2007/0096311, 2007/0096295, 2005/0095835, 2005/0087861,2005/0085016, 2005/0082654, 2005/0082653, 2005/0067688, and U.S. patentapplication No. 61/609,293, and PCT applications nos. PCT/US2012/024018and PCT/IB2012/000381.

Components of MEMS actuators in accordance with alternative embodimentsare described at U.S. Pat. Nos. 7,972,070, 8,014,662, 8,090,252,8,004,780, 7,747,155, 7,990,628, 7,660,056, 7,869,701, 7,844,172,7,832,948, 7,729,601, 7,787,198, 7,515,362, 7,697,831, 7,663,817,7,769,284, 7,545,591, 7,792,421, 7,693,408, 7,697,834, 7,359,131,7,785,023, 7,702,226, 7,769,281, 7,697,829, 7,560,679, 7,565,070,7,570,882, 7,838,322, 7,359,130, 7,345,827, 7,813,634, 7,555,210,7,646,969, 7,403,344, 7,495,852, 7,729,603, 7,477,400, 7,583,006,7,477,842, 7,663,289, 7,266,272, 7,113,688, 7,640,803, 6,934,087,6,850,675, 6,661,962, 6,738,177 and 6,516,109; and at US publishedpatent applications nos. 2010/030843, 2007/0052132, 2011/0317013,2011/0255182, 2011/0274423, and at U.S. unpublished patent applicationSer. Nos. 13/302,310, 13/247,938, 13/247,925, 13/247,919, 13/247,906,13/247,902, 13/247,898, 13/247,895, 13/247,888, 13/247,869, 13/247,847,13/079,681, 13/008,254, 12/946,680, 12/946,670, 12/946,657, 12/946,646,12/946,624, 12/946,614, 12/946,557, 12/946,543, 12/946,526, 12/946,515,12/946,495, 12/946,466, 12/946,430, 12/946,396, 12/873,962, 12/848,804,12/646,722, 12/273,851, 12/273,785, 11/735,803, 11/734,700, 11/848,996,11/491,742, and at USPTO-Patent Cooperation Treaty applications (PCTS)nos. PCT/US12/24018, PCT/US11/59446, PCT/US11/59437, PCT/US11/59435,PCT/US11/59427, PCT/US11/59420, PCT/US11/59415, PCT/US11/59414,PCT/US11/59403, PCT/US11/59387, PCT/US11/59385, PCT/US10/36749,PCT/US07/84343, and PCT/US07/84301, which are all incorporated byreference.

All references cited above and below herein are incorporated byreference, as well as the background, abstract and brief description ofthe drawings, and U.S. patent application Ser. Nos. 12/213,472,12/225,591, 12/289,339, 12/774,486, 13/026,936, 13/026,937, 13/036,938,13/027,175, 13/027,203, 13/027,219, 13/051,233, 13/163,648, 13/264,251,and PCT application WO2007/110097, and U.S. Pat. Nos. 6,873,358, andRE42,898.

The following are also incorporated by reference as disclosingalternative embodiments:

U.S. Pat. Nos. 8,055,029, 7,855,737, 7,995,804, 7,970,182, 7,916,897,8,081,254, 7,620,218, 7,995,855, 7,551,800, 7,515,740, 7,460,695,7,965,875, 7,403,643, 7,916,971, 7,773,118, 8,055,067, 7,844,076,7,315,631, 7,792,335, 7,680,342, 7,692,696, 7,599,577, 7,606,417,7,747,596, 7,506,057, 7,685,341, 7,694,048, 7,715,597, 7,565,030,7,636,486, 7,639,888, 7,536,036, 7,738,015, 7,590,305, 7,352,394,7,564,994, 7,315,658, 7,630,006, 7,440,593, and 7,317,815, and U.S.patent application Ser. Nos. 13/306,568, 13/282,458, 13/234,149,13/234,146, 13/234,139, 13/220,612, 13/084,340, 13/078,971, 13/077,936,13/077,891, 13/035,907, 13/028,203, 13/020,805, 12/959,320, 12/944,701and 12/944,662, and United States published patent applications serialnos. 2012/0019614, 2012/0019613, 2012/0008002, 2011/0216156,2011/0205381, 2012/0007942, 2011/0141227, 2011/0002506, 2011/0102553,2010/0329582, 2011/0007174, 2010/0321537, 2011/0141226, 2010/0141787,2011/0081052, 2010/0066822, 2010/0026831, 2009/0303343, 2009/0238419,2010/0272363, 2009/0189998, 2009/0189997, 2009/0190803, 2009/0179999,2009/0167893, 2009/0179998, 2008/0309769, 2008/0266419, 2008/0220750,2008/0219517, 2009/0196466, 2009/0123063, 2008/0112599, 2009/0080713,2009/0080797, 2009/0080796, 2008/0219581, 2009/0115915, 2008/0309770,2007/0296833 and 2007/0269108.

Auto-focus features may be included in a camera or camera module asdescribed at US published patent application no. 2012/0075492 and/orU.S. application Ser. Nos. 12/944,701, 12/944,703, 13/020,805,13/077,891 and 13/077,936.

Features described at U.S. application Ser. Nos. 13/028,203, 13/028,205and 13/028,206 are incorporated by reference and may also be used inalternative embodiments to register images captured that have global orlocal rotation between them and/or to discern the motion of the cameramodule and/or one or more objects in a captured scene.

It will be appreciated that the illustrated embodiment is provided forexemplary purposes only and that many variations of the implementationare possible. For example, some functionality shown as being implementedin one module could be migrated to other modules.

In the illustrated embodiment, tiles have been described as rectangularand defined by four nodes. In another embodiment, non-rectangular tilesare defined by 3 or more nodes; and indeed the local grid need not bedefined by a uniform array of tiles. Instead, these could in certainapplications be non-uniform.

The invention is not limited to the embodiment(s) described herein butcan be amended or modified without departing from the scope of thepresent invention, as set forth in the appended claims and structuraland functional equivalents thereof.

What is claimed is:
 1. A method of generating a digital video imageusing a lens positioned in front of an image sensor array, includingproviding a lens having a sufficiently wide field of view (WFOV), andpositioned sufficiently near to the sensor array, that the image fieldof the lens is so curved at the sensor array that different regions ofthe image field are substantially in focus on the sensor array fordifferent positions of the lens relative to the sensor array, the methodfurther comprising: (a) selecting a desired region of interest in theimage field of the lens, (b) adjusting the position of the lens relativeto the sensor array so that the selected region of interest is broughtsubstantially into focus on the sensor array, (c) capturing and storingthe image on the sensor array of the substantially in-focus selectedregion of interest, (d) at least partially correcting the storedsubstantially in-focus image for field-of-view distortion due to saidWFOV lens, (e) displaying the corrected image, and (f) cyclicallyrepeating steps (a) to (e).
 2. The method claimed in claim 1, whereinadjusting the position of the lens relative to the sensor arraycomprises adjusting the axial distance of the lens from the sensorarray.
 3. The method claimed in claim 1, wherein the selected region ofinterest is one of a plurality of sub-regions of a larger region ofinterest which cannot be brought substantially into focus as a whole onthe sensor array, and wherein step (b) comprises adjusting the positionof the lens in respect of each said sub-region to bring that sub-regionsubstantially into focus on the sensor array, and step (c) comprisescapturing and storing each said sub-region, and combining the storedsub-region images to form a substantially in-focus image of the largerregion of interest.
 4. The method claimed in claim 1, wherein the lenscomprises a MEMS-actuated lens.
 5. The method claimed in claim 1 whereinstep (a) comprises selecting a plurality of regions of interest andrepeating steps (b) to (e) for each region of interest.
 6. The method ofclaim 1 wherein step (a) comprises automatically detecting a desiredregion of interest within the complete image field.
 7. The method ofclaim 6 wherein said image comprises a plurality of image planesincluding an infra-red (IR) image plane and wherein said automaticallydetecting is performed on said IR plane of said image.
 8. The method ofclaim 7 wherein said steps (c) to (e) are performed on visible colourplanes of said image.
 9. The method of claim 7 wherein saidautomatically detecting is based on face detection or gesture detectionor both.
 10. The method of claim 1 wherein said selecting is based onvideo or audio analysis or both.
 11. A digital image acquisition devicefor generating a digital video image comprising: a lens positioned infront of an image sensor array, the lens having a sufficiently widefield of view (WFOV), and being positioned sufficiently near to thesensor array, that the image field of the lens is so curved at thesensor array that different regions of the image field are substantiallyin focus on the sensor array for different positions of the lensrelative to the sensor array, the device including a processor arrangedto iteratively: (a) select a desired region of interest in the imagefield of the lens, (b) adjust the position of the lens relative to thesensor array so that the selected region of interest is broughtsubstantially into focus on the sensor array, (c) capture and store theimage on the sensor array of the substantially in-focus selected regionof interest, (d) at least partially correct the stored substantiallyin-focus image for field-of-view distortion due to said WFOV lens, and(e) display, transmit or store the corrected image.
 12. The deviceclaimed in claim 11, wherein adjusting the position of the lens relativeto the sensor array comprises adjusting the axial distance of the lensfrom the sensor array.
 13. The device claimed in claim 11, wherein theselected region of interest comprises one of a plurality of sub-regionsof a larger region of interest which cannot be brought substantiallyinto focus as a whole on the sensor array, and wherein step (b)comprises adjusting the position of the lens in respect of each saidsub-region to bring that sub-region substantially into focus on thesensor array, and step (c) comprises capturing and storing each saidsub-region, and combining the stored sub-region images to form asubstantially in-focus image of the larger region of interest.
 14. Thedevice claimed in claim 11, wherein the lens comprises a MEMS-actuatedlens.
 15. The device claimed in claim 11 wherein step (a) comprisesselecting a plurality of regions of interest and repeating steps (b) to(e) for each region of interest.
 16. The device of claim 11 wherein step(a) comprises automatically detecting a desired region of interestwithin the complete image field.
 17. The device of claim 16 wherein saidimage comprises a plurality of image planes including an infra-red (IR)image plane and wherein said automatically detecting is performed onsaid IR plane of said image.
 18. The device of claim 17 wherein saidsteps (c) to (e) are performed on visible colour planes of said image.19. The device of claim 17 wherein said automatically detecting is basedon face detection or gesture detection or both.
 20. The device of claim11 wherein said selecting is based on video or audio analysis or both.